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(57) Abstract: A sound reproduction system comprises headphones (1 1). Said headphones comprise means for generating sound 
(4, 5) and microphones (6, 7) (i.e. means for recording sound). Further the system comprises filter means (8,9) to filter a signal such 
that the sound produced simulates external sound sources. The system comprises a feed-back and control system (10) in which a 
signal (r,(k), r r (k)) from the microphone (6, 7) is used to set the settings W XL (k), W^k) of the filter means (8,9). 
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Headphones with integrated microphones. 10/5^8489 



The invention relates to a sound reproducing system comprising headphones 
with sound generating means and means to control the sound signal generated by said 
headphone sound generating means to simulate external sound sources. 

5 The invention also relates to a headphone for a sound reproducing system. 

Headphones are used in and for audio equipment such as (mobile) CD-players, 
but also in call-in centres. 

10 The headphones comprise a means for generating sound (usually a small 

loudspeaker). A recorded sound signal (voice or music) is sent to the headphone(s) and sound 
generators inside the headphone generate a sound. The listener will, however, perceive the 
generated sound as being generated inside or very near the head (which in fact it is) unless the 
sound signal is adapted. Such a sound is perceived to be unnatural. It is known to process the 

15 signals such that the perception of the sound signal by the listener is such that he/she believes 
to hear external sound sources i.e. perceive a more natural sound. To achieve this the signals 
are processed through a filter which filter is set to alter the characteristics of the signal such 
that the sound generated near or within the head simulates an (or more than one) external 
sound source(s). An important aspect in this respect is the transfer characteristics of sound by 

20 an external source to the head and pinnae itself, the so-called Head Related Transfer Function 
(HRTF), i.e. the manner in which sound becomes attenuated and altered by the head and 
pinnea itself before it actually is heard. Attempts to process the signals taking into account the 
HRTF to obtain external source simulation are known from J. Acoust. Soc. Am. 85(2), pages 
858-878, F.L. Wightman and D. Kistler, Feb. 1989: 'Headphone simulation of free-field 

25 listening I and IT. 

Such attempts however do not always prove to be successful. The HRTF are 
dependent on the actual shape and form of the head and the ear and differ substantially from 
one person to another. Furthermore head movements complicate matters as they also influence 
the sound perception. It has been known from for instance Japanese patent application JP 
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08/079900 A to provide the headphones with measuring devices to measure the distance 
between the ears, the height of the head and head movements. Although such measurements 
can be used to improve the sound reproduction the results leave room for improvement. The 
HTRF is a strongly individual one which can only be approximately determined using the 
result of such measurement. Likewise the effect of head movements can only be 
approximately determined. 

It is an object of the invention to provide a sound system as described in the 
opening paragraph with improved sound reproduction. 

To this end the system is characterized in that the headphones are provided with 
microphones, and the means to control, comprise or are coupled to means to regulate the 
sound production by the headphone sound generating means such that a signal registered by 
the microphones is substantially zero when at least one external sound source is operative in 
response to a signal and means to record the results of said regulating to influence external 
source simulating sound generation in the headphones and/or means to regulate the sound 
production by the headphone sound generating means such that the difference between a 
signal registered by the microphones and a known signal is substantially zero and means to 
record the results of said regulation to influence external sound simulating sound generation in 
the headphones. 

Each headphone is provided with a microphone. Said microphone which is 
located near or preferably in the ear registers the sound generated by the headphone sound 
generating means as well as in one aspect of the invention by the at least one external source. 
The system comprises means to regulate the sound production by the headphone sound 
generating means such that the microphone registers a substantially zero signal when 
simultaneously at least one external source in response to a signal and the headphone sound 
generating means are active. The headphone then generates a, as far as the human perception 
is concerned, same auditive signal but of opposite sign as the external source(s). The system 
has means to record the results of the regulation. Thereafter, when the external source(s) (is) 
are shut off, or removed altogether, the sound perceived by the listener is the same as that for 
the external sources. The signal registered by the microphone will be equivalent to that when 
only the source would be operative. The relation between a signal sent to the source such as a 
loudspeaker and the signal sent to the headphone sound generating means to simulate such an 
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external source is then known. The data from the above mentioned regulation are used for 
regulation of the sound signal to the headphones in such manner that the external source is 
simulated. 

5 The relation between 

a signal sent to an external source 
a signal to the headphone sound generating means and 
a microphone signal 

are thus measured. Such measurement does, however, not only give the relation between 
10 signals a (external source signal) and b (equivalent headphone signal), but also between 

signals b (headphone signal) and c (microphone signal) and signals a (external source signal) 
and c (microphone signal). These known relations can also or separately be used in another 
aspect of the invention as follows. 

15 Once for a 'standard head' or in fact for any head the relations between signals 

a, b and c have been established, it is not in all circumstances, i.e. for other heads necessary to 
make further use external source with signal a. It suffices to know (and this is known) the 
microphone signal c corresponding to a particular external source signal a to regulate 
headphone signal b if needed. When the headphone sound generating means 'truly' (signal b) 

20 simulate an external source (signal a) a particular microphone signal (signal c) should be 
registered. This is the case on the 'standard head'. However, when the headphone is put on 
another head the HRTF will be different and the same signal b sent to the headphone sound 
generating means will generate a microphone signal c' different from said particular 
microphone signal c because of the different HRTF. The system has means to regulate the 

25 signal b sent to the headphone sound generating means (to b') in such manner that signal c' is 
equal to signal c, record the regulation data and use the regulation data for further sound 
production to simulate external source(s). 

It is remarked that whereas in embodiments the headphone sound generating 
means and the microphone will be often separate elements, in some embodiments the 

30 headphon sound generating means (headphone loudspeakers) may double in function as the 
microphone, especially when such headphone sound generating means are placed inside the 
ear channel. 

Preferably the system also comprises means to store the regulation data for a 
specific person. 
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This enables regulation data to be kept and coupled to a specific user. The next 
time said user uses the system an incoming signal is filtered in the 'right' or at least 'nearly 
right' manner. 

These and other objects of the invention will be apparent from and elucidated 
5 with reference to the embodiments described hereinafter. 

In the drawings: 



10 Fi S- 1 illustrates schematically how to generate from two real sound sources a 

third so-called phantom source. 

Fig. 2 illustrates schematically a system in accordance with the invention. 

Fig. 3 illustrates schematically a further embodiment of a system in accordance 
with the invention. 

15 Fi S- 4 illustrates yet a further embodiment of a system in accordance with the 

invention. 

Fig. 5 illustrates a still further embodiment of a system in accordance with the 

invention. 

Fig. 6 illustrates another aspect of the invention 

20 Fi S s - 7A to ?E illustrate several embodiments of a headphone for a system in 

accordance with the invention. 

Fig. 8 illustrates schematically how the headphone sound generating means 
may be also the microphone. 

25 

The figures are schematic and not drawn on scale. 

Figure 1 shows a head of a person 1 with two ears 2 and 3. Two real 
loudspeakers LS, (loudspeaker-left) and LS r (loudspeaker-right) are present in a room. With 
30 these loudspeakers it is meant to generate a sound as if a sound signal V r is generated by a 
loudspeaker LS P at some other point in space. 

To calculate which signals have to be generated by the real loud speakers LS, 
and LS r to give the person 1 the impression that the sound he hears is generated by a 
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W X R=(W pl \/ lr -W pr W rT )/CW tr W rl -W ll W rT ) 

The filter functions which in this simplified model have been described actually 
have to be determined for each frequency thus actually for each frequency a filter function 
Wxr and WxLhas to be determined and fixed and used. With the proper filter functions Wxr 
and W X l, the listener hears the 'phantom source 1 LS P . Thus with two loud-speakers a 
'phantom' sound source at a sound can be generated which, to the listener seems to come from 
another location than the actual location of the loud speakers LSl and LSr. This perception is 
dependent on the accuracy of the transfer functions (in this application sometimes also called 
'filters' or 'filter settings') W X l and Wxr 

The filters W X L^nd Wxr are difficult to determine because the transfer 
functions Wn, W^, W^ and W t from the loudspeakers LSi and LS r to the ear" are difficult to 
determine. The transfer function for the real loudspeakers to some extent can be calculated and 
/or measured for a 'standard head', but in reality each head and each headphone is different 
and thus a transfer function is always more or less appropriate but never really good. The 
transfer functions for the phantom source can only be estimated or theoretically derived. 
Especially for the higher frequencies, the transfer functions are difficult to determine because 
of the shape of the head and the ear canal. In short the Head Related Transfer function, HRTF, 
is a highly individual one. 

The transfer function needs to be calculated and the calculation introduces 

errors. 

For each frequency the transfer function has to be determined, which either 
requires a large calculation effort and such calculation in itself may be a source of error or 
necessitates the use of average transfer functions for a band of frequencies, which also 
introduces errors. 

All transfer functions are to some extent dependent not just on the relative 
positions of the sound sources (real or phantom) and the ears, but also on other factors, such as 
objects near the sources or ears which may reflect or alter the sound waves and thus influence 
the transfer functions. 

Thus there is a need to improve the sound reproduction. 
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Figure 2 illustrates a preferred embodiment of a system in accordance with the 

invention. 

The system comprises two headphones each of which is provided with a 
microphone 6, 7. Each of the headphones has sound generating means 4, 5. A signal x(k) is 
5 relayed to the means 4, 5 through filter means (i.e. modulation means) 8, 9 having filter setting 
W XL (k) and WxR(k). In previous systems the filters 8, 9 were fixed filters (as in figure 1) and 
thus the settings W XL (k) and W XR (k) were fixed. These fixed filters were usually set to be 
equivalent to an 'average head' in an 'average room'. The signals after the filters are indicated 
with e,(k) and 6,0c). The signals are e,(k)= x(k)*W XL (k)and e r (k)= x(k)* W^k). In the system 

10 in accordance with the invention microphones 6 and 7 are present in or near the headphones 
and generate a signal n(k) and r r (k). The signals n(k) and r 2 (k) are due to the sum of the sound 
generated by the external source and the headphone. These signals r,(k)and r r (k) are fed to 
comparison and regulation means 10 which also have an input for signal x(k) and an output to 
filter means 8, 9 to adapt or regulate settings W XL (k) and W^k). It will be noticed that in 

15 figure 2 only the transfer functions W n and W„. are shown. This will be explained below. 

A signal x(k) is supplied to the sound source PL and signals §i(k) and e r (k) are 
supplied to the sound generating means 4 and 5. The signals rj(k) and r r (k) are fed to the 
regulating means 10. This regulating means influences the settings of the filters W XL (k) and 

20 WxR(k) (and thereby the signals ei (k)= x(k)*W XL (k) and e r (k) = x(k)*W XR (k)) until the 

microphone signals n(k) and r r (k) (and this preferably for each or for a chosen set or selection 
of frequencies) become substantially zero. This may be done by a step-wise manner, i.e. one or 
more parameters (one or more of the settings W XL (k) or W XR (k)) is (are) changed, it is then 
checked whether the signal n(k) is increased or decreased, if it is increased, the parameter(s) is 

25 (are) changed in the opposite sense, if it is decreased, the parameter(s) is (are) changed in the 
same sense. This process is repeated until the signals n(k) and r r (k) are substantially zero. For 
more details of such methods reference is made to e.g. 'Adaptive Filter Theory* by Simon 
Haykin, Prentice Hall, Upper Saddle River, ISBN 0-13-322760-X. It is remarked that in 
general the less parameters have to be taken into account in such methods, the better the result 

30 is and the faster the result can be achieved. When the microphone signal r r (k) and rj(k) are 

substantially zero the listener hears nothing. The resulting values for filter settings W XL (k) and 
W XR (k) are thereby determined. These filter settings can be for instance tables in a computer 
data base. When the source PL is shut off or removed, the listener will hear a sound which by 
the listener is perceived to come from said source PL. Thus the listener hears a 'phantom 
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source' at the position of source PL. If the system is to be used for one person only, such 
tables could be the only one to use, but preferably the system comprises a means 
(schematically indicated by input I in figure 2) to store established settings WxiXk) and 
WxrOO for the filters 8, 9 and pair the settings to data identifying the person. The next time the 
5 same person uses the system the filter will then be set right or at least nearly right for said 
person provided information identifying the person is given to the system. In practice tables 
are for instance stored in a computer data base paired with a name or number identifying the 
person. 

Compared to previous methods and devices the results are better and much 

10 more reliable, a much more 'natural' sounding and better 'localized' phantom source is heard 
by the listener. An advantage over fixed filters is that Wxx and W X r can easily, faster and with 
much greater accuracy be determined and be adapted for different locations and for different 
persons. For instance if head transfer functions are calculated with fixed Filters often 
parameters such as an average height and width of an average head are used, such parameters 

15 are sometimes useless or may even give clearly wrong results if the person in question carries 
some head ware such as a hat or for instance has a size head substantially different from the 
average head. Even the height of the hair may be of importance in this respect. Furthermore 
more parameters than inter ear distance and head height may be of importance for the HRTF. 
The present invention does not suffer from these shortcomings but gives reliable results for 

20 each person, irrespective of the size and shape of the head and or ear and/or whether said 
person wears a hat because all these factors do not play a role due to the microphone. 
Furthermore the cross transfer functions (W r i and W fr ) are, due to the nearness of the source 4, 
5 to the ear 2, 3, negligible or in any case very small. This enables in preferred embodiments, 
as e.g. shown in figure 2, to further greatly simplify the calculation, thus removing a source of 

25 error. In formula form it holds 

Wxl=(W p i)/(W„) 
WxR^WpJ/CW,) 

30 

These formulae are much simplified compared to formulae for phantom sound 
generation using loudspeakers with fixed filters. For each ear the filter functions are only 
dependent on two, not six, transfer functions. In fact the determinations of the filter settings 
W X r and W X l are independent. The measurement at the left (right) ear suffices to determine 
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Wxx(k) (WxR(k)). This enables faster (less response time) and much better determination of 
WxLand Wxr. Furthermore the response of the acoustic paths of the headphones is very short 
(thus further shortening response time). Furthermore extraneous influences such as the shape 
of a room and objects in a room on the transfer functions Wn, (and W r j, W^) is not present 
in headphone sound reproduction. As a consequence when tests were done with a system as 
schematically shown in figure 2 to see what the perceived difference would be between the 
real loudspeaker and a phantom loudspeaker, the location of the phantom loudspeaker was 
correct for both an anechoic room (a room in which sound reflection is reduced to a minimum) 
and a listening room (a room with normal sound reflection). These results were much better 
than for known systems using fixed filters. As an alternative (and this may be in particular of 
importance for source at a relatively large distance) instead of working with a signal coming 
from each microphone, the sum (ri(k)+r r (k)) and difference (n(k)-r r (k)) of these two signals 
could also be used. If the sum and the difference are zero, both signals are zero. Usually W n 
and W„- are nearly equal (symmetric), and at large distances from the source W pJ and W pr are 
also not to much different. These facts are preferably used to simplify the calculations. It is 
remarked that in figure 2 the different filter means (8, 9) and regulation means (10) are drawn 
separately to increase clarity. They may, and preferably are all integrated in one device. In 
certain circumstances, for instance a nearly symmetrically arranged fixed position of the 
source only one microphone could be used. The data of said one microphone would then 
suffice. 

Figure 3 illustrates a further embodiment of a system in accordance with the 
invention. Two loudspeakers PLj and PL 2 are used. For both loudspeakers the transfer 
functions WxLand WxRcan be determined in the manner as described above. This can be done 
in the following manner. First loudspeaker PLi is activated and microphone signals are made 
zero. The filter settings W X L(k) and W X R(k) for said loudspeaker are determined. Thereafter 
loudspeaker PLi is deactivated and loudspeaker PL 2 is activated to determine filter settings 
W' XL (k) and W , X R(k) for loudspeaker PL 2 . The filter functions for both loud speakers having 
been determined, the system is capable of reproducing any mix of the two sound sources PLi 
and PL 2 with a very natural sound, i.e. stereo.sound. 

For a signal x(k) sent to loudspeaker PLj and simultaneously a signal y(k) sent 
to loudspeaker PL 2 the signals to the headphone sound generating means are: 



e»(k)= x(k)*W XL (k) + y(k)*W XL (k) and 
e,(k)= x(k)*W XR (k) + y(k)*W' XL (k) 
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When more than twc sources are to be simulated the signals to the more than 
two sources could for instance be written as a vector and the filter settings for the different 
sources could be written in matrix form. Multiplication of the vector (for the sources) with the 
matrix (for the settings) will generate the signals ej(k) and e r (k). The matrix itself is 
5 determined by measurements and may be different for different persons and different rooms. 

A further embodiment of the system in accordance with the invention is shown 
in figure 4. Having established the transfer functions W X l and W X r respectively W'xl and 
W'xr for two loudspeakers PLi and PL2, this knowledge can be used to 'create* using for 
instance geometrical principles more phantom sound sources, for instance phantom 

10 loudspeakers PL 3 and PL4. Using for instance thereafter the above technique of vector-matrix 
multiplication a 'surround sound* may be created. The problem with trying to do so using 
fixed filters lies, as already explained amongst others in the very individual Head Related 
Transfer Functions and also from local circumstances such as reverberation in a room. Starting 
from two known sources one can using geometry and/or standard techniques calculate the 

15 transfer function for the phantom sources PL 3 and PL* in so far as geometry is concerned but 
not or much less the other influences. In a system in accordance with the invention said 
difficulty is resolved for the main part, since use is made of actual measurements on an actual 
head with actual headphones (thus taking into account the relevant HRTF) and in an actual 
room (thus at least partly taking into account the reverberation in the room) resulting in 

20 transfer functions which take these influences in account giving much better rendition of 
phantom sources. 

A yet further embodiment is shown in figure 5. The headphones (or at least one 
of them, or the connection between the headphones) comprise means to measure the position 
in regards of the two sources PL, and Pl 2 and/or some fixed reference point. Such means can 

25 be for instance infra-red sources which are sensed by sensors in or near the sources PLi and 

PL 2 or infra red sources in or near PL] and PL 2 which are sensed by sensors in the headphones. 
Such means may also comprise means generating and sensing ultra-sound. In this example the 
two 'real* loudspeakers are positioned at either side of a television set 51. Near or at at least 
one headphone an emitter of a signal or sensor for localization signals is present and an 

30 stationary part of the system comprises a sensor or emitter for localization signals. 

As explained before the transfer functions are determined using the 
microphones 6 and 7 and the two sources PL, and PL 2 are turned off they are then audible in 
the headphones as 'phantom sources*. The transfer functions to simulate these two external 
sources PLi and PL 2 then include the individual HRTF and room related factors. Knowing the 
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microphone 6 near or in the inner ear. In figure 7C the headphone 1 1 and microphone 6 are 
separate devices but both placed in or near the ear. The outputs signal of the sound generating 
means are led to a jack 72, the output signal of the microphone to a separate jack 7L In figure 
7 E both output signals are led to a single jack 73 which has two separate ports 75 and 76 
5 through which the signals may be transferred to a part of the system. This embodiment is the 
most preferred embodiment, because one single jack is necessary. The part of the sound 
system in which the jack will be inserted may be provided with means to pick up the signals. 
Such a jack can be a standard jack, but for the extra output, likewise the part of the sound 
system in which the jack will be inserted may be standard, but for the possibility of registering 

10 the signal from the microphone. This enables 'standard* equipment, at least as far as the user is 
concerned to be used. The sound system will be able to operate with 'normal headphone' (in 
which case there will be no microphone signal, but will be able to register whether or not a 
headphone in accordance with a system of the invention is used, and if so, operate in 
accordance with the invention. 

15 Figure 7D illustrates that the signal (n(k), r r (k) or any combination of derivative 

of or data representing said signals) from the microphone can be relayed wireless as well as by 
a separate plug. 

It will be clear that within the framework of the invention many variations are 

20 possible. 

For instance in the above given examples the microphone is shown as an 
element separate from the other elements. In embodiments the headphone sound generating 
means themselves may be used as microphone. Figure 8 illustrates very schematically how 
this can be done. Headphone sound generating means 81 comprises or are coupled to or with a 

25 means 82 to drive a membrane 83 to generate sounds. Said system is supplied with a signal I in 
via an input 84. The headphone sound generating means also comprise means 85 (which may 
have some, most or even all building elements common to means 82) with an output 86 which 
generate a signal Io Ul corresponding to the movement of the membrane. A means 87 for 
regulating the signal l in has an input for signal Io Ut and regulates Ij n such that W becomes 

30 substantially zero when an external source generates a sound I. In those circumstances the 
sound pressure at the position of the membrane is zero, thus it is silent. Preferably, for these 
embodiments, i.e. for the embodiments wherein the headphone sound generating means 
double in function as microphones, the headphone sound generating means are in operation 
located inside the ear. 
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In short the invention can be described as follows: 

A sound reproduction system comprises headphones (1 1). Said headphones 
comprise means for generating sound (4, 5) and microphones (6, 7)(i.e. means for recording 
sound). Further the system comprises filter means (8,9) to filter a signal such that the sound 
produced simulates external sound sources. These filter means comprise filter setting date 
WxrQc), WxiXk). The system comprises a feed-back and control system (10) in which a signal 
(n(k), r r (k)) from the microphone (6, 7) is used to set the settings Wxi/k), WxR(k) of the filter 
means (8,9). The signal can be used by making it zero (when an external source is used) 
(n(k)=0, see figure 3) or by comparing the microphone signal and a gauge signal zero (r"(k)- 
r'(k)=0, see figure 6) such that the two are substantially the same. 

It is remarked that systems are known, for instance for use in very high noise 
environments such as airports, to cancel noise. In some of such systems a microphone inside 
the headphone is used. The headphone sound generating means make a counternoise to cut out 
or at least reduce strongly all noise within a certain frequency bandwidth. The idea behind 
such systems is that by eliminating the usually low frequency noise, the noise to signal ratio 
between the noise and the usually more high frequency communication sounds signals is 
increased. Such systems, however, do not simulate external sources nor are the microphone 
signals used to set filter settings. 
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CLAIMS: 



1. Sound reproducing system comprising headphones (11) with sound generating 
means (4, 5) and means to control (8, 9) the sound signal generated by said head phone sound 
generating means to simulate external sound sources (PLj, PL^PLo, PL 4 ), characterized in that 
the headphones (1 1) are provided with microphones (6, 7), and the means to control comprise 

5 or are coupled to means to regulate (10) the sound production by the headphone sound 

generating means such that a signal (n(k), r r (k)) registered by the microphones is substantially 
zero when at least one external sound source (PLi, PL 2 ) is operative in response to a signal 
(x(k)) and means (8,9) to record the results of said regulating to influence external source 
simulating sound generation in the headphones and /or the means to control comprise or are 
10 coupled to means (10) to regulate the sound production by the headphone sound generating 
means such that a comparison between a signal registered by the microphones (r'(k)) and a 
known or calculated signal (r"(k)> r ,,f (k), r""(k)) show said two signals to be substantially 
the same and means (8,9) to record the results of said regulation to influence external sound 
simulating sound generation in the headphones. 

15 

2. Sound reproducing system as claimed in claim I, characterized in that it 
comprises means (10, 1) to register regulation data (W X L(k), W X p(k)) for a person. 
Sound reproducing system as claimed in claim 1, characterized in that the headphone is 
provided with a sound transporting means (12). 

20 

3. Sound reproducing system as claimed in claim 3, characterized in that the 
sound transporting means comprise a tube (12). 

4. Sound reproducing system as claimed in claim 1 , characterized in that the 
25 microphone (6) is integrated in a headphone insertable inside the ear. 



5. Sound reproducing system as claimed in claim 1, characterized in that the 

headphone sound generating means is a microphone. 
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6. Sound reproducing system as claimed in claim 1, characterized in that the 
system comprises means to establish the relative position of the headphones and a stationary 
element of the system. 

7. Sound reproducing system as claimed in claim 6, characterized in that near or at 
at least one headphone an emitter of a signal or sensor for localization signals is present and 
the fixed part comprises a sensor or emitter for localizing signals. 



8. 



Headphone for use in a sound system as claimed in any of the preceding claims. 
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FIG. 2 
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